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Preface 


The Defense Personnel Security Research Center (PERSEREC) is conducting 
research to test the feasibility of a system for operating aperiodic personnel security 
reinvestigations in lieu of the current system of reinvestigation every five years. The 
system is called Automated Continuing Evaluation (ACES). This report is the first in a 
series. Additional research is now underway to further assess the feasibility of ACES and 
other personnel security continuing evaluation procedures that are based upon similar 
strategies. 


The Office of Personnel Management (OPM) data on which this study is based 
provides solid support for PERSEREC and others to further test and debate the merits of 
modifying existing personnel security policies to introduce a system of aperiodic 
reinvestigations and automated continuing evaluations. These evaluations involved 
automated checks of security-relevant databases, such as criminal history, credit, foreign 
travel, and large-currency transactions. This approach would reduce security risk by 
detecting more cases involving issues of serious security concern and by detecting those 
cases earlier. Furthermore, it would substantially reduce demands on investigative 
resources. This would be accomplished by applying more investigative resources to the 
relatively small number of cases where they are needed the most and fewer resources to 
cases where they are needed the least. Using this approach, full-scale reinvestigations 
would be triggered based on factors such as the person’s level of access, the time elapsed 
since the last investigation, whether issues were detected in the last investigation, and the 
seriousness and number of new issues detected by automated checks of security-relevant 
databases. Some of the investigative resources that are freed up could be applied to 
reduce the length of time it takes to complete initial personnel security investigations. 


While the report’s findings should be considered preliminary, they do support 
what could be a significant change to the Federal personnel security system: the adoption 
of an a-periodic reinvestigation cycle coupled with more frequent automated checks of 
security-relevant databases. 

James A. Riedel 
Director 
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Vili 


Executive Summary 


A retrospective design was employed in this study to estimate the number of full- 
scale reinvestigations that would have been triggered under the proposed Automated 
Continuing Evaluation System (ACES) in 11,065 Office of Personnel Management 
(OPM) periodic reinvestigation cases for people holding high-level Federal security 
clearances. An assessment was also made of the number of serious issue cases identified 
under the present system that would have been missed by ACES. Under the present 
system, full-scale reinvestigations are required to be conducted on all personnel holding 
high-level clearances who have not been investigated for five years. Under ACES, 
computerized security-related information (e.g., criminal history, foreign travel, and 
credit database files) would be regularly checked. Full-scale reinvestigations in individual 
cases could be triggered at any time or might never be initiated based upon the results of 
the electronic checks, and consideration of other risk-management factors. Findings 
indicate that a majority of the security clearance holders in the OPM reinvestigation 
sample would not have needed full-scale reinvestigations based upon the ACES criteria. 
In addition, no serious issue cases and very few moderately serious issue cases detected 
by the present system would have been missed had the recommended ACES procedures 
been in effect for those cases. ACES is likely to detect some serious issue cases currently 
being missed because the personnel quit before their periodic reinvestigations were 
initiated. Consequently, ACES is likely to detect more serious issue cases than the present 
system. It would also detect those serious issue cases sooner and at less cost than the 
current periodic reinvestigation approach. These findings should be considered 
preliminary in nature due to methodological limitations inherent in this assessment. They 
are, however, supportive of the continued development of ACES and other personnel 
security continuing evaluation procedures that are based upon similar strategies. 
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Introduction 


Background 


Single Scope Background Investigation (SSBI) — Periodic Reinvestigations are 
required every five years for people holding the highest-level Federal security clearances 
and accesses (e.g., TOP SECRET, Q, SCI). 


In the last decade, two national commissions have criticized the security clearance 
procedures. (Commission on Protecting and Reducing Government Secrecy, 1997; Joint 
Security Commission; 1994). The Joint Security Commission (1994) recommended that 
current reinvestigation policies be refined to increase efficiency and cost effectiveness, 
noting that an aperiodic reinvestigation interval would offer a greater deterrent to adverse 
security-related behavior and provide agencies with more flexibility to focus their 
resources on priority investigations (p. 46). 


The Commission on Protecting and Reducing Government Secrecy (1997) stated: 


Greater attention needs to be directed toward making continuing evaluation 
programs more effective. For example, using existing public and private data 
bases — with the express advance permission of the individual under review — to 
periodically scan for criminal history, as well as credit, travel, and business 
history, normally would provide more accurate information at less cost than 
standard field reinvestigations. 


Personnel security professionals could monitor the behavior of cleared personnel 
on a continuing basis in a more effective, cost-efficient, and nonintrusive manner. 
Given the evidence that there is little likelihood of catching spies through the 
current standard investigative or reinvestigative process, better continuing 
assessment programs could enhance the probability of deterring or identifying 
espionage activities. Most of the information needed is already available in 
existing databases; private industry experiences suggest that efforts to utilize 
automation to access such data can be very cost-effective as well as productive. 
Nevertheless, because some automated tools can be expensive, a cost-benefit 
assessment should be completed prior to utilizing them. 


Resources should be focused on those individuals in the most sensitive positions 
or where there is some evidence of suspect behavior; in an era of diminishing 
resources and frequent budget cuts, more effective continuing assessment can be 
accomplished only by concentrating on the areas of greatest vulnerability. In 
addition, those holding what are identified as the most sensitive positions could be 
subjected to more frequent, “in house” reviews similar to the personnel reliability 
programs used by the Defense and Energy Departments, as described above. 
These measures provide a cost-effective way to monitor and assess employees 
with greater regularity and frequency, but without necessarily having to direct 
additional resources toward the traditional field investigation (pp. 86-87). 


Automated Continuing Evaluation System (ACES) 


An Automated Continuing Evaluation System (ACES) prototype, which is 
presently under development in the Department of Defense, embodies the aforementioned 
Commission recommendations. Instead of evaluating on a five-year cycle whether 
cleared personnel have been engaging in behaviors of serious security concern, ACES 
would check relevant databases far more frequently using a rationally based aperiodic 
reinvestigation strategy. Full-scale reinvestigations would be triggered based on such 
factors as evidence of behaviors of security concern, individuals’ access to particularly 
sensitive information or materials, length of time since the last full-scale investigation, 
and on a random basis. A more detailed explanation of the proposed ACES criteria used 
for determining when full-scale aperiodic reinvestigations would be triggered is presented 
in Appendix A. 


The Defense Personnel Security Research Center (PERSEREC) recently 
completed a database matching pilot study that assessed the feasibility and value of 
acquiring computerized data from 15 different governmental and private vendor 
databases for use in personnel security investigations (Chandler, Timm, Massey, & 
Zimmerman, 2000). Statistical matches and “live” record checks on actual Defense 
Security Service cases helped identify which of those databases provide the most useful 
information. Other studies have also addressed the productivity of certain sources that 
will be included under ACES, such as credit reports and Treasury Department large 
currency transaction report data (Carney, 1996; Defense Manpower Data Center, 1987a, 
1987b, 1988, 1989). 


Purpose 


The purpose of the present study is to assess two measures that may affect the 
viability of ACES, the number of serious issue cases detected and the number of full- 
scale reinvestigations triggered. The number of cases with serious issues identified by the 
most comprehensive Office of Personnel management (OPM) periodic reinvestigations 
(PRs) is compared to the estimated number that would have been identified had the 
ACES database checks and case expansion criteria been applied to those same cases. If 
ACES is unable to identify most of the serious cases, adopting the new procedure in lieu 
of the present approach could pose a serious threat to national security. If ACES triggers 
more full-scale reinvestigations than are conducted under the present system, the program 
may not be cost effective. 


Method 


The method section is divided into two phases. Before the main research 
questions could be addressed, it was necessary to establish an adjudication-based 
categorization of the seriousness of cases. The first phase describes how those categories 
were created. The second phase describes the approach and procedures used for 
addressing the primary research questions. 


The number of full-scale reinvestigations that would have been triggered by 
ACES in those cases is compared to the total number of PR cases in the sample. 


Phase 1 - Establishing Adjudication-Based Designated Seriousness Categories for 
Cases 


OPM uses the following seriousness code continuum to rate cases: G - no issues, 
A-— minor issues, B — moderate issues, C — substantial issues, D — major issues. It also 
employs other codes, such as: E — other, J — major issues to which the subject made 
admissions, W — issues that may be relevant depending on the mission of the 
organization, and R — no actionable issues. 


Subjects 


The overall OPM case seriousness code and the adjudication action taken by the 
agency that requested the investigation were obtained for 49,260 initial background 
investigations (OPM Investigation Types 20-39) and 29,007 periodic reinvestigations 
(OPM Investigation Types 11-13 & 18). No personal identifiers were provided. Cases 
used for this analysis were closed by OPM from 10/01/95 to 9/30/98. Adjudication action 
taken by the agency requesting the investigation was available in PIPS for 82.3% of the 
initial background investigation, and 49.6% of the periodic reinvestigation cases were 
closed during that period. 


Procedure 


The adjudicative outcomes for the cases included in this phase of the study were 
cross tabulated by the above OPM seriousness codes in order to reduce ambiguity and 
apparent overlap in OPM seriousness categorizations. Based upon the percentages of 
cases that received favorable adjudications within each category of the OPM seriousness 
codes, the case seriousness categories designated for analytical purposes were as follows: 
Codes G and R were designated None; Codes E, W, and A were designated Very Minor; 
Code B was designated Minor; Code C was designated Moderate; and Codes J and D 
were designated Major. To distinguish these codes from OPM seriousness codes, they 
will be identified as “Designated Seriousness Categories” throughout the remainder of 
this document. 


Tables 1 and 2 show the percentages of cases favorably and negatively adjudicated 
as well as the percentages of subjects who resigned prior to the adjudication decision. 
Table 1 reports those values for the OPM seriousness codes within each of the new, 
adjudication-based, Designated Serious Categories for Initial Full-Scope Background 
Investigations. Table 2 presents that information for Periodic Reinvestigations. 


Table 1 
Designated Seriousness Categories for OPM Seriousness Codes Based Upon Agency 
Adjudication Action on Initial Full-Scope Background Investigations* 


OPM Favorable Subject Resigned Negative Designated 
Seriousness Adjudication (OPM Code 4) Adjudication Seriousness 
Code (OPM ras 1,2,3) % (OPM — 5,6,7,8) Category 
n 0 0 ; 
G_ 9,612 98.6 1.2 0.0 None 
R_ 12,336 98.0 1.7 ot 
E 4,026 95.3 3.9 Ve Minor 
A _ 3,739 94.5 4.7 
W 12,693 94.5 4.2 Ve Minor 
B 3,191 88.5 9.2 ia 
C 2,084 80.2 13.7 2.0 Moderate 
D_1,557 54.2 ae hee ee ee ee ee 
J 22 45.5 22.7 9.1 Major 


Note. N=49,260. 
@ Only cases where adjudication action was reported to OPM were included. Peiceritapes across rows do 
not total 100% because some other type of action was taken by the agency in a small number of cases. 


Table 2 
Designated Seriousness Categories for OPM Seriousness Codes based upon Agency 
Adjudication Action on Periodic Reinvestigation*® 


OPM Favorable Subject Resigned Negative Designated 
Seriousness Adjudication (OPM Code 4) Adjudication Seriousness 
Code (OPM Codes 1,2,3) (OPM Codes 5,6,7,8) Category 


None 
None 


7 T A: Minor 


Note. N=29,007 

2 Only cases where adjudication action was reported to OPM were included. Percentages across rows do 
not total 100% because some other type of action was taken by the agency in a small number of cases. No 
row for “J” level cases appears because only six cases with outcome information were coded as “J” -- four 
were favorably adjudicated, one resigned, and one fell into the category for “another type of action was 
taken.” 


Phase 2 - Estimating the Impact of ACES 
Subjects 


Investigation outcome information was obtained on 11,686 OPM Single-Scope 
Background Investigation - Periodic Reinvestigation cases (OPM Investigation Type 18) 


that were closed with a completed investigation between 10/01/97 and 9/30/99. No 
personal identifiers were included in the data set provided. Cases issued a “K” 
seriousness code by OPM were excluded from the analyses. “K” codes are issued when 
OPM officials do not provide an overall case seriousness rating for administrative 
reasons. A total of 621 cases in the data set were excluded from the analyses due to a “K” 
designation instead of a normal seriousness code in either the subject’s initial 
investigation, the reinvestigation, or both. This left 11,065 cases for the primary analyses. 


Approach 


The adjudication-based Designated Seriousness Categories established in Phase 
One that applied to completed full-scope SSBI reinvestigations served as benchmarks for 
assessing the ability of ACES to identify cases with serious issues. That measure was 
chosen because the full-scope SSBI reinvestigation includes the monitoring measures 
used by ACES (e.g., completion of a personnel security questionnaire, national agency 
database, and credit and local agency checks) as well as several other investigative 
procedures not used by ACES unless a full-scope investigation is triggered (e.g., subject, 
developed reference, neighbor, and co-worker interviews). It is assumed that one five- 
year check of the databases monitored at year five by ACES would yield the same total 
amount of security-related information gleaned from five annual checks covering that 
same period. For example, if an individual had three arrests posted in a criminal history 
database during a covered five-year interval, both ACES and SSBI-PR checks should 
detect all three arrests for that period. ACES, however, has the ability to detect the arrests 
earlier than the SSBI-PR checks. 


Due to the study’s retrospective design, the time periods covered by the ACES 
estimates were fixed at the same interval covered by the subjects’ actual OPM periodic 
reinvestigations. Given that full-scale reinvestigations triggered under ACES would 
encompass the same number and type of face-to-face interviews as normal SSBI-PRs, 
ACES full-scope reinvestigations should detect the same issues of security concern as 
SSBI-PRs that cover the same time period. Therefore, for the period between the 
subject’s last completed full-scope investigation and the subject’s most recent PR, the 
outcome of the SSBI-PR was considered an appropriate quality benchmark for assessing 
ACES. ACES can meet the SSBI-PR serious issue detection output for the time period 
covered by a completed PR, but theoretically it should never be able to outperform it 
within that same interval, because the same sources would have been checked whenever 
full-scope reinvestigations were triggered. 


In addition to assigning an overall case seriousness score in its personnel security 
cases, OPM requires its contract investigators to designate the type and seriousness of the 
issues developed from each type of source of information used during investigations. 
According to officials at OPM, investigators evaluate the information found at each 
source independently from the information found at other sources. Consequently, it is 
possible to separate the issues detected by the sources that would have been routinely 
monitored by ACES from those that would have only been obtained if a full-scale 
reinvestigation had been triggered. Values used by ACES to determine whether or not a 
full-scope reinvestigation would have been triggered were calculated by considering the 


presence and level of issues of security concern developed from the personnel security 
questionnaire, national agency checks, credit check, local agency check, and basic 
employment check. The values were also calculated without consideration of the 
information obtained during the basic employment check in order to assess the relative 
utility of including that source of information in ACES. 


Procedure 


In the data provided by OPM, each category of investigative source used in the 
investigation was rated as follows: 0-—no issues were identified by that source, 1—a 
relatively minor issue was identified, or 2—either a more serious issue or multiple issues 
were found. If one or more issues were identified by any of the sources monitored by 
ACES, and the overall case seriousness code was not rated as G, R, or K, that case was 
categorized as having identified at least a minor issue under ACES conditions. If the sum 
of the OPM category scores for the measures monitored by ACES was 2 or higher, and 
the overall OPM case seriousness code was designated as B, C, J, or D, that case was 
categorized as having identified at least a moderate issue under ACES conditions. If the 
sum of the OPM item scores for the measures monitored by ACES was 4 or higher, and 
the overall case seriousness code was designated as C, J, or D, that case was categorized 
as having identified a major issue under ACES conditions. Another way C, J, or D level 
cases were categorized as identifying major issues under ACES conditions was if the sum 
of the OPM item scores for the measures monitored by ACES was 3, and one of the OPM 
item ratings was 2. 


As described in the preceding paragraph, when determining issue seriousness 
according to ACES, the OPM overall case seriousness codes were used as upper-range 
limits for the level of issues identified by ACES, but not as lower-range limits. The 
rationale for using the overall OPM seriousness score as an upper-range limit is that the 
level of information of security concern that can be collected from a subset of the 
investigative sources in theory should never exceed the level that would have been 
collected from the entire set. The overall OPM seriousness score, however, was not used 
as a lower limit because it is clearly possible to collect less information of security 
concern from a subset of the investigative sources than would have been collected from 
the entire set. For example, ACES might find no issues of security concern in “major 
issue” OPM cases because the sources that provided that information would not have 
been checked by ACES (e.g., exclusively from the neighborhood interviews). However, 
ACES should never be able to identify “major issues” in cases covering the same period 
of investigation in which a full-scope SSBI PR found no issues. 


Criteria used for identifying cases that would have been selected by ACES to receive 
a full-scope reinvestigation 


A primary function of ACES is to trigger full-scale aperiodic reinvestigations. In 
this study, only five of the seven criteria listed in Appendix A were used to identify cases 
that would have been selected by ACES to receive a full-scope reinvestigation. Points 
that would have been issued at random or to people who have access to especially 
sensitive information or materials were not taken into consideration in the analyses. It is 


not known which of the people who completed the OPM reinvestigations had especially 
sensitive accesses. The impact of random point assignment on both resources and serious 
case detection will vary depending upon the level of chance selected by the end-user. The 
effects due to random point assignment can be easily estimated by the reader for any level 
of chance considered. The five criteria that were included in the estimates are presented 
below in the form of questions. Each time the response to a question is “yes” in a given 
case another point was awarded. Consequently, the range of points awarded in the 
analyses was from 0 to 5. Cases receiving three or more points would have been slated to 
receive full-scope reinvestigations under ACES. (See also Appendix A.) 


e Has it been five years or more since that person’s last full-scale investigation? 
Since all of the investigations in the OPM sample were PRs, it is believed they all 
would have met this criterion. Therefore, one point was issued in every such case. 


e Was an issue of security concern identified by that person’s initial background 
investigation? Ifan OPM overall case seriousness code of E, A, W, B, C, D, or J 
was issued in the person’s initial background investigation, one point was issued. 


e Was an issue or set of issues of at least minor security concern detected by the 
ACES sources? If so, one point was issued. 


e Was an issue or set of issues of at least moderate security concern detected by the 
ACES sources? If yes, one point was issued in addition to the point issued for 
having at least minor issues detected. 


e Was an issue or set of issues of major security concern detected by the ACES 
sources? If yes, one point was issued in addition to the two points issued for 
having at least minor and moderate issues detected. 


Results 


Table 3 presents the total number of subjects within each Designated Seriousness 
Category who resigned before completion of their investigations for either initial 
background investigations or periodic reinvestigations. The percentages of people whose 
initial background investigations and periodic reinvestigations resulted in negative agency 
adjudication actions by Designated Seriousness Category are presented in Table 4. 


Table 3 also depicts two easily discernable relationships: 1) as the level of 
seriousness of issues detected increased, resignation rates rose and 2) the resignation rates 
were considerably higher among people undergoing their initial investigation than among 
people undergoing their periodic reinvestigation. The first relationship suggests that if 
people thought there was a high likelihood that issues of major security concern would 
surface during their next periodic reinvestigation, it is reasonable to believe they may be 
more inclined to quit before that reinvestigation was initiated. Given that ACES would 
monitor security-relevant databases between full-scale personnel security investigations, 
it would undoubtedly identify some of those cases that now escape detection due to that 
limitation. While it is comforting to deduce that some of the people engaging in acts of 
serious security concern probably quit their positions to escape identification, under the 
present approach there is a five-year window of vulnerability between investigations, 
during which time a person in a position of trust can do an enormous amount of damage. 


Not surprisingly, the same relationships previously noted for resignations prior to 
adjudication were found for negative adjudication actions taken by agencies (see Table 4). 
They were 1) as the level of seriousness of the issues detected increased, negative 
adjudication rates rose and 2) the negative adjudication rates were considerably higher 
among people undergoing initial investigations than among people undergoing periodic 
reinvestigations. What was surprising was how low the negative adjudication rate was 
for reinvestigations involving major issues. Among the factors that probably contributed 
to that low rate were 1) people opting to resign in lieu of receiving a negative 
adjudication, 2) agencies’ reluctance to lose trained, functioning employees, and 3) a 
desire to treat incumbents as humanely as possible, including helping them work through 
personal problems whenever possible. Given the low negative adjudication rates for PRs, 
it is clear that much of the value derived from reinvestigations is in deterring people from 
engaging in negative behaviors and as a means for getting them into treatment when 
appropriate. 


Table 3 


for Either Initial Background Investigations or Periodic 
Reinvestigations by Designated Seriousness Category 


Moderate 
Major 


Overall 


285/2,084 


394/1,579 


2,166/49,260 


Table 4 


25.0 


Designated Seriousness Initial Background Periodic Reinvestigations 
Category Investigations 
n % n % 
None 325/21,948 1.5 57/17,916 0.3 


Very Minor 868/20,458 4.2 93/9,211 1.0 
Minor 294/3,191 9.2 40/1,158 3.5 


26/456 


31/266 


4.4 247/29,007 


Comparison of the Percentage of People Who Resigned Prior to Adjudication 


0.9 


Comparison of the Percentage of People Whose Initial Background 
Investigations and Periodic Reinvestigations Resulted in Negative 
Adjudication Action by Designated Seriousness Category 


Designated Seriousness Initial Background Periodic Reinvestigations 
Category Investigations 
n % n % 
None 11/21,948 0.1 1/17,916 0.0 
Very Minor 50/2,0458 0.2 10/9,211 0.1 
Minor 23/3,191 0.7 3/1158 0.3 
Moderate 41/2,084 2.0 2/456 0.4 


8 A 
279/49,260 0.6 17/29,007 0.1 


A comparison of the percentages of people in each Designated Seriousness 
Category is presented in Table 5 for the initial background investigation and periodic 
reinvestigation cases addressed in Tables 1-4, as well as for the FY 98 and 99 SSBI PR 
subset of those cases used to estimate the impact of ACES. 


Both lower-issue case rates and lower-issue seriousness levels for incumbents 
than for people undergoing initial background investigations are documented in Table 5. 
Deterrence, prior screening, and maturity probably all contributed to those effects. 


Table 5 
Comparison of the Percentage of People in Each Designated Seriousness Category 
for FY97, 98, and 99 Initial Background Investigations and Periodic Reinvestigations 
and for FY98 and 99 SSBI Periodic Reinvestigations 


Designated Initial Background Periodic SSBI Periodic 
Seriousness Category Investigations Reinvestigations Reinvestigations 
(FY 97, 98, & 99) (FY 97, 98, & 99) (FY 98 & 99) 
n % n % n % 
None 44.6% 61.8% 52.0% 
21,948/49,260 17,916/29,007 5,756/11,065 
Very Minor 41.7% 31.8% 41.7% 
20,458/49,260 9,211/29,007 4615/11,065 
Minor 6.5% 4.0% 3.4% 
3,191/49,260 1,158/29,007 376/11,065 
Moderate 4.2% 1.6% 1.7% 
2,084/49,260 456/29,007 187/11,065 
Major 3.2% 0.9% 1.2% 
1,579/49,260 266/29,007 131/11,065 


Two tables are presented that compare the number of points that ACES would 
have issued for each Designated Seriousness Category. Table 6 includes issues raised by 
employment interviews in the ACES scores; Table 7 does not include issues raised by 
that source. In both tables, if the ACES score was 3 or greater, a full-scope 
reinvestigation would have been triggered. Given that a full-scope reinvestigation 
initiated under ACES would have included the same investigative components as a full- 
scope OPM reinvestigation, all of the issues detected by the OPM reinvestigation would 
have been discovered in cases with ACES scores of 3 or more. 
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Table 6 
Distribution of OPM Periodic Reinvestigation Cases by ACES Score and Designated 
Seriousness Category — Employment Issues Included in the ACES Score? 


| ACES Score | Score Designated Seriousness Categ gO. Based on All OPM Sources 


ACES 
Sources 
2 2188 1477 se | 
3 0 ie ae aa pe 
4 0 a 
5 0 ee 
% Selected for 
Full-Scope PR 0% 63.7% 96.8% 97.9% 
by ACES 
4 Cases with an ACES score of 3 or more would have received a full-scope reinvestigation. Consequently, 


all of the issues detected by the OPM reinvestigation would have been found in the cases that met or 
exceeded that threshold. 


Table 7 
Distribution of OPM Periodic Reinvestigation Cases by ACES Score and Designated 
Seriousness Category — Employment Issues Not Included in the ACES Score? 


ACES Score _| Designated Seriousness Category Based on All OPM Sources 


(Based only on | None Very Minor Minor Moderate Major 
ACES 
Sources 
2 2188 1401 14 
3 : 0 2929 75 
4 0 0 278 
5 0 0 0 
% Selected for 
Full-Scope PR 0° 63.4% 93.9% 
by ACES 


4 Cases with an ACES score of 3 or more would have received a full-scope reinvestigation. Consequently, 
all of the issues detected by the OPM reinvestigation would have been found in the cases that met or 
exceeded that threshold. 


As depicted in Table 6, when employment sources were included in the ACES 
score, 100% of the OPM reinvestigation cases involving major issues would have been 
fully investigated under ACES. Conversely, none of the OPM cases that found no issues 
of security concern would have received full investigations under ACES. This is 
important because 52% (5756 out of 11,065) of the OPM SSBI reinvestigation cases 
found no issues of security concern during the course of the reinvestigation. The 
percentages of cases that would have received full-scope reinvestigations under ACES 
rose as the seriousness of the issues detected increased (63.7% for very minor, 96.8% for 
minor, and 97.9% for moderate issues of security concern). 
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Overall, 67.3% (7445 out of 11,065) of the OPM reinvestigation cases in Table 6 
had ACES scores under 3 even when issues identified through employment sources were 
included. In other words, approximately two thirds of those OPM cases would not have 
received full-scope reinvestigations under ACES. The non-expansion rate is slightly 
larger for the cases presented in Table 7, which did not consider issues identified through 
employment-related checks (67.6% or 7477 out of 11,065 cases). Once again, the 
percentage of cases that would have received full-scope reinvestigations under ACES 
rose as the seriousness of the issues detected increased (63.4% for very minor, 93.9% for 
minor, 95.7% for moderate, and 96.9% for major issues of security concern). 


The value of including an employment-related check as part of the measures 
routinely monitored by ACES can be seen by comparing Tables 6 and 7. Issues of major 
security concern found in four cases would have been missed without taking into 
consideration the information identified during the course of the OPM basic employment 
check. This is not surprising considering the amount of time employees spend at work 
and the formal and informal feedback mechanisms for detecting employee misconduct 
that typically operate in the workplace. In addition to finding out from security managers 
and/or supervisors whether subjects have engaged in any behaviors of security concern, 
they could be asked whether the subject still needs a high level clearance and whether the 
person has access to especially sensitive information. Under ACES, subjects having 
access to especially sensitive information are more likely to receive full-scope 
reinvestigations. 


Discussion 


This study employed a retrospective design entailing analysis of existing 
computer-readable records from previously completed OPM periodic reinvestigation 
cases. Using that approach allowed the analyses to be performed quickly and at virtually 
no additional cost. However, it also limited the analyses to those that could be performed 
based exclusively on the cases and data at hand. Cleared personnel who quit their jobs 
before the PRs included in the analyses were initiated were de facto excluded from the 
study. This point is illustrated in Figure 1. 


Prior Behavior of Periodic 
— — 


Year 1 Years 2-5 Year 5 
Subject could be included in a retrospective study based on PR data. 


Prior Behavior of Person 
— > 
Year 1 Years 2 - 4 Year 4 
Subject could not be included in a retrospective study based on PR data. 


Figure 1. Behavior of Security Concern 
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The primary analyses were based exclusively on closed, complete periodic 
reinvestigation cases. Undoubtedly in some of the reinvestigation cases initiated, 
significant issues of security concern were found and called to the subjects’ attention, 
which probably occasionally resulted in their resigning before their reinvestigations were 
completed. Those cases were also de facto excluded from the analyses. The obvious 
problem associated with including prematurely closed cases is that not all of the 
investigation elements were completed. In certain prematurely terminated cases not all of 
the ACES database checks would have been finished. In those cases, it is impossible to 
determine from the data at hand whether additional issues of security concern would have 
been discovered had the remaining ACES checks been conducted. Similarly, it is 
impossible to discern from the records at hand whether cases, which appeared “clean” 
when terminated, would have later turned into “issue cases” had the remaining sources 
been checked. Given those limitations and the lower drop-out levels found for PRs than 
for initial investigations, no attempt was made to determine whether the ACES protocol 
would have been more or less effective in dealing with prematurely terminated cases than 
it was with the completed cases. 


Another limitation stemming from the retrospective design is that the interval 
between investigations was fixed at the same period reflected in the OPM period 
reinvestigations. Although this interval is ideal for predicting the impact of applying the 
ACES protocol to cases currently due for a periodic reinvestigation, it does not permit 
identifying how ACES would have performed inside that five-year interval. For example, 
it is not known in how many of the cases in which ACES would have triggered a full- 
scope reinvestigation would it have also triggered a second or maybe even a third 
reinvestigation during that period if the person did not change his/her behavior. It is also 
unknown how much earlier ACES would have detected issues of major security concern 
compared to the current approach. However, none of the cases that ACES did not select 
for full-scope reinvestigations, based upon record reviews covering the five previous 
years, would have been selected for full-scope reinvestigations had annual or even more 
frequent checks been run covering that same interval. Therefore, after correcting for 
random point assignment and access sensitivity, the estimated percentage of cases that 
would have not been selected for a full-scope PR should be fairly accurate for the time 
intervals considered. 


The ACES scores reflected in the primary analyses took into consideration the 
results from the subjects’ initial investigations. If an issue of security concern was raised 
during that investigation the subject received one point toward the threshold for receiving 
a full-scope reinvestigation. The rationale for including that element in ACES is that 
people with prior issues are more likely to have issues detected in the future. It probably 
would have been preferable to take into consideration the outcome from all the subjects’ 
prior full-scope investigations, especially the most recent ones, instead of only their first 
investigations. Subjects with multiple clean reinvestigations, including their most recent 
clean reinvestigations, are probably less likely to have issues of security concern surface 
than subjects who do not meet those criteria. 
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While the OPM data indicate that approximately two thirds of the reinvestigation 
cases for high-level security clearances and access would not have received a full-scope 
reinvestigation under ACES, that figure does not represent the likely savings. As 
previously noted, a small number of the people under ACES would have probably 
received more than one full-scope reinvestigation during the five-year period. Some of 
the people who received less than three points in the analyses depicted in Tables 6 and 7 
would have required a full-scope reinvestigation due to receiving additional points for 
access sensitivity and/or random assignment which were not included in the estimates. 
While the ACES electronic checks are much less expensive than the face-to-face 
interviews that are included in full-scope investigations, many of those electronic checks 
are not free and would be done at least once a year. People under the present system who 
quit between the five-year reinvestigation intervals and receive no follow-up investigation 
were not included in the two-thirds figure’s denominator. It is strongly recommended 
that before any savings from adoption of ACES are considered for removal from existing 
budgets that they be fully known and have already been achieved. 


Conclusion 


Based upon the preliminary analyses conducted in this study, it appears that 
ACES has the potential to dramatically decrease the number of full-scale PRs that need to 
be initiated with practically no decrease in the ability to identify cases containing serious 
issues. Traditional, fixed-interval PRs miss serious issues occurring between 
investigations whenever people having those issues quit prior to their upcoming PR. 
Therefore, it is anticipated that for people holding TS/SCI clearances, ACES will detect 
more serious issue cases, and will detect them sooner and at less cost, than the traditional 
PR approach. Furthermore, ACES will help close a five-year window of vulnerability in 
which issues of serious security concern are allowed to escape detection and intervention. 
Additional research is now underway to further assess the feasibility of ACES and other 
personnel security continuing evaluation procedures that are based upon similar 
strategies. 
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Appendix A 


Proposed ACES Criteria for Selecting Cases for Full-Scale 
Reinvestigations 


A-2 


PROPOSED ACES CRITERIA FOR SELECTING CASES FOR FULL-SCALE 
REINVESTIGATIONS 


e Was an issue of security concern warranting adjudication consideration (e.g., it 
was not a case where the alleged misconduct was proven to be unfounded or 
mistakenly attributed to the subject) identified by that person’s last full-scale 
investigation? If yes, one point is issued. 


e Has it been five years or more since that person’s last full-scale investigation? If 
yes, one point is issued. 


e Does the person have access to especially sensitive classified information or 
materials? If yes, one point is issued. 


e Has an issue or set of issues of at least minor security concern been detected by 
the database monitoring since the subject’s last full-scale reinvestigation? If yes, 
one point is issued. 


e Has an issue or set of issues of at least moderate security concern been detected by 
the database monitoring since the subject’s last full-scale reinvestigation? If yes, 
one point is issued in addition to the point issued for having at least minor issues 
detected. 


e Has an issue or set of issues of major security concern been detected by the 
database monitoring since the subject’s last full-scale reinvestigation? If yes, one 
point is issued in addition to the two points issued for having at least minor and 
moderate issues detected. 


e Was the person assigned an additional point at random at the completion of 
his/her last full-scale investigation? (The chances of being issued one point is 1 


out of 20.) If yes, one point is issued. 


e Each affirmative answer receives one point. Three or more points trigger a full- 
scale reinvestigation. (Range = 0 to 7 points.) 


A-3 


